Method for processing image based on scene recognition of image and electronic device therefor

ABSTRACT

An electronic device and a method for setting image quality are provided. An electronic device includes a camera, a memory, and a processor configured to obtain, using the camera, a plurality of images for one or more external objects, identify a region of interest and a background region using one or more images of the plurality of images, recognize a first object included in the region of the interest, identify a type of the background region by recognizing a second object included in the background region, determine a scene corresponding to the one or more images, based on the recognized first object and the identified type of the background region, and adjust at least one of an image quality setting associated with the camera or an image quality setting associated with the one or more images using a specified image quality setting corresponding to the determined scene.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims priority under 35 U.S.C. §119(a) to Korean Patent Application No. 10-2018-0092707, filed on Aug.8, 2018, in the Korean Intellectual Property Office, the entiredisclosure of which is incorporated herein by reference.

BACKGROUND 1. Field

The disclosure relates generally to a method for processing an imagebased on a recognized scene of the image and an electronic device forperforming the method.

2. Description of Related Art

An image-based service may be performed based on information extractedby analyzing an image. To improve user experience, various image-basedservices may be provided. For example, a retrieving service, an imageediting service, an image conversion service, or an image qualityrecommendation service, which is based on image analysis, may beprovided to the user.

A conventional electronic device may set the quality of an image, basedon the analysis of the image. For example, the electronic device maycorrect an obtained image by changing an image parameter for the image.The electronic device may also recommend an optimal photographingenvironment, based on the analysis of the image. The electronic devicemay provide a user with an image which is obtained depending on an imagequality setting adjusted using a photographing setting valuecorresponding to the image.

A conventional electronic device may also provide a method for adjustingimage quality settings of various images. For example, the electronicdevice may provide various filters for adjusting the image qualitysettings of obtained images. In another example, the electronic devicemay provide various photographing modes for adjusting an image qualitysetting of an image. However, complexity may be increased due to theselection of the filter or the photographing mode.

The electronic device may also perform image analysis based on raw data.However, for image analysis based on the raw data, the image may beanalyzed based on pixel values of the image, rather than the contextinformation of the image. Therefore, according to the image analysisbased on the raw data, the electronic device may provide results that donot correspond to the context of the user.

SUMMARY

Aspects of the disclosure are designed to address at least theabove-described problems and/or disadvantages and to provide at leastthe advantages described below.

Accordingly, an aspect of the disclosure is to provide a method forcorrecting an image corresponding to a context of a user.

In accordance with an aspect of the disclosure, an electronic device isprovided, which includes a camera, a memory, and a processor operativelyconnected with the camera and the memory. The processor is configured toobtain, using the camera, a plurality of images for one or more externalobjects, identify a region of interest and a background region by usingone or more images of the plurality of images, while obtaining theplurality of images, recognize a first object included in the region ofthe interest, identify a type of the background region by recognizing asecond object included in the background region, determine a scenecorresponding to the one or more images, based on the recognized firstobject and the identified type of the background region, and adjust atleast one of an image quality setting associated with the camera or animage quality setting associated with the one or more images by using aspecified image quality setting corresponding to the determined scene.

In accordance with another aspect of the disclosure, an electronicdevice is provided, which includes a camera, a display, a processoroperatively connected with the camera and the display, and a memoryoperatively connected with the processor. The memory includesinstructions, which when executed, cause the processor to obtain animage, identify an object from a partial region of the obtained image,identify a first tag corresponding to the identified object, identify asecond tag corresponding to an entire region of the obtained image,identify a scene tag corresponding to the obtained image based onreliabilities of the first tag and the second tag, and adjust an imagequality setting parameter of the camera using an image quality settingparameter corresponding to the identified scene tag.

In accordance with another aspect of the disclosure, a method isprovided for setting image quality in an electronic device. The methodincludes obtaining an image; identifying an object from a partial regionof the obtained image; identifying a first tag corresponding to theidentified object; identifying a second tag corresponding to an entireregion of the obtained image; identifying a scene tag corresponding tothe obtained image, based on reliabilities of the first tag and thesecond tag; and adjusting an image quality setting parameter of a cameraof the electronic device by using an image quality setting parametercorresponding to the scene tag.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 illustrates an electronic device in a network, according to anembodiment;

FIG. 2 illustrates a camera module, according to an embodiment;

FIG. 3 illustrates an electronic device, according to an embodiment;

FIG. 4 is a flowchart illustrating a method for determining scenerecognition, according to an embodiment;

FIG. 5 is a flowchart illustrating a method for determining a type of abackground image, according to an embodiment;

FIG. 6 is a flowchart illustrating a method for identifying an object,according to an embodiment;

FIG. 7 is a flowchart illustrating a method for determining a scene,according to an embodiment;

FIG. 8 is a flowchart illustrating a method for subsidiarily determininga scene, according to an embodiment;

FIG. 9 is a flowchart illustrating a method for determining an imagequality setting, according to an embodiment;

FIG. 10 illustrates a method for recognizing a scene, according to anembodiment;

FIG. 11 illustrates an image for determining a reliability, according toan embodiment;

FIG. 12 illustrates an image for recognizing an object, according to anembodiment;

FIG. 13 illustrates an image for determining a scene, according to anembodiment;

FIG. 14 illustrates an image for determining a scene, according to anembodiment; and

FIG. 15 is a flowchart illustrating a method for adjusting an imagequality setting, according to an embodiment.

DETAILED DESCRIPTION

Various embodiments of the disclosure are described with reference toaccompanying drawings. Those skilled in the art should understand thatfollowing embodiments and terminology used herein are not to limit thetechnology disclosed herein to a specific embodiment, but includemodifications, equivalents, and/or alternatives of the describedembodiments.

FIG. 1 illustrates an electronic device 101 in a network environment 100according to an embodiment.

Referring to FIG. 1, the electronic device 101 in the networkenvironment 100 may communicate with an electronic device 102 via afirst network 198 (e.g., a short-range wireless communication network),or an electronic device 104 or a server 108 via a second network 199(e.g., a long-range wireless communication network). According to anembodiment, the electronic device 101 may communicate with theelectronic device 104 via the server 108. According to an embodiment,the electronic device 101 may include a processor 120, memory 130, aninput device 150, a sound output device 155, a display device 160, anaudio module 170, a sensor module 176, an interface 177, a haptic module179, a camera module 180, a power management module 188, a battery 189,a communication module 190, a subscriber identification module (SIM)196, or an antenna module 197. In some embodiments, at least one (e.g.,the display device 160 or the camera module 180) of the components maybe omitted from the electronic device 101, or one or more othercomponents may be added in the electronic device 101. In someembodiments, some of the components may be implemented as singleintegrated circuitry. For example, the sensor module 176 (e.g., afingerprint sensor, an iris sensor, or an illuminance sensor) may beimplemented as embedded in the display device 160 (e.g., a display).

The processor 120 may execute, for example, software (e.g., a program140) to control at least one other component (e.g., a hardware orsoftware component) of the electronic device 101 coupled with theprocessor 120, and may perform various data processing or computation.According to one embodiment, as at least part of the data processing orcomputation, the processor 120 may load a command or data received fromanother component (e.g., the sensor module 176 or the communicationmodule 190) in volatile memory 132, process the command or the datastored in the volatile memory 132, and store resulting data innon-volatile memory 134. According to an embodiment, the processor 120may include a main processor 121 (e.g., a central processing unit (CPU)or an application processor (AP)), and an auxiliary processor 123 (e.g.,a graphics processing unit (GPU), an image signal processor (ISP), asensor hub processor, or a communication processor (CP)) that isoperable independently from, or in conjunction with, the main processor121. Additionally or alternatively, the auxiliary processor 123 may beadapted to consume less power than the main processor 121, or to bespecific to a specified function. The auxiliary processor 123 may beimplemented as separate from, or as part of the main processor 121.

The auxiliary processor 123 may control at least some of functions orstates related to at least one component (e.g., the display device 160,the sensor module 176, or the communication module 190) among thecomponents of the electronic device 101, instead of the main processor121 while the main processor 121 is in an inactive (e.g., sleep) state,or together with the main processor 121 while the main processor 121 isin an active state (e.g., executing an application). According to anembodiment, the auxiliary processor 123 (e.g., an image signal processoror a communication processor) may be implemented as part of anothercomponent (e.g., the camera module 180 or the communication module 190)functionally related to the auxiliary processor 123.

The memory 130 may store various data used by at least one component(e.g., the processor 120 or the sensor module 176) of the electronicdevice 101. The various data may include, for example, software (e.g.,the program 140) and input data or output data for a command relatedthereto. The memory 130 may include the volatile memory 132 or thenon-volatile memory 134.

The program 140 may be stored in the memory 130 as software, and mayinclude, for example, an operating system (OS) 142, middleware 144, oran application 146.

The input device 150 may receive a command or data to be used by othercomponent (e.g., the processor 120) of the electronic device 101, fromthe outside (e.g., a user) of the electronic device 101. The inputdevice 150 may include, for example, a microphone, a mouse, or akeyboard.

The sound output device 155 may output sound signals to the outside ofthe electronic device 101. The sound output device 155 may include, forexample, a speaker or a receiver. The speaker may be used for generalpurposes, such as playing multimedia or playing record, and the receivermay be used for an incoming calls. According to an embodiment, thereceiver may be implemented as separate from, or as part of the speaker.

The display device 160 may visually provide information to the outside(e.g., a user) of the electronic device 101. The display device 160 mayinclude, for example, a display, a hologram device, or a projector andcontrol circuitry to control a corresponding one of the display,hologram device, and projector. According to an embodiment, the displaydevice 160 may include touch circuitry adapted to detect a touch, orsensor circuitry (e.g., a pressure sensor) adapted to measure theintensity of force incurred by the touch.

The audio module 170 may convert a sound into an electrical signal andvice versa. According to an embodiment, the audio module 170 may obtainthe sound via the input device 150, or output the sound via the soundoutput device 155 or a headphone of an external electronic device (e.g.,an electronic device 102) directly (e.g., wiredly) or wirelessly coupledwith the electronic device 101.

The sensor module 176 may detect an operational state (e.g., power ortemperature) of the electronic device 101 or an environmental state(e.g., a state of a user) external to the electronic device 101, andthen generate an electrical signal or data value corresponding to thedetected state. According to an embodiment, the sensor module 176 mayinclude, for example, a gesture sensor, a gyro sensor, an atmosphericpressure sensor, a magnetic sensor, an acceleration sensor, a gripsensor, a proximity sensor, a color sensor, an infrared (IR) sensor, abiometric sensor, a temperature sensor, a humidity sensor, or anilluminance sensor.

The interface 177 may support one or more specified protocols to be usedfor the electronic device 101 to be coupled with the external electronicdevice (e.g., the electronic device 102) directly (e.g., wiredly) orwirelessly. According to an embodiment, the interface 177 may include,for example, a high definition multimedia interface (HDMI), a universalserial bus (USB) interface, a secure digital (SD) card interface, or anaudio interface.

A connecting terminal 178 may include a connector via which theelectronic device 101 may be physically connected with the externalelectronic device (e.g., the electronic device 102). According to anembodiment, the connecting terminal 178 may include, for example, a HDMIconnector, a USB connector, a SD card connector, or an audio connector(e.g., a headphone connector),

The haptic module 179 may convert an electrical signal into a mechanicalstimulus (e.g., a vibration or a movement) or electrical stimulus whichmay be recognized by a user via his tactile sensation or kinestheticsensation. According to an embodiment, the haptic module 179 mayinclude, for example, a motor, a piezoelectric element, or an electricstimulator.

The camera module 180 may capture a still image or moving images.According to an embodiment, the camera module 180 may include one ormore lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to theelectronic device 101. According to one embodiment, the power managementmodule 188 may be implemented as at least part of, for example, a powermanagement integrated circuit (PMIC).

The battery 189 may supply power to at least one component of theelectronic device 101. According to an embodiment, the battery 189 mayinclude, for example, a primary cell which is not rechargeable, asecondary cell which is rechargeable, or a fuel cell.

The communication module 190 may support establishing a direct (e.g.,wired) communication channel or a wireless communication channel betweenthe electronic device 101 and the external electronic device (e.g., theelectronic device 102, the electronic device 104, or the server 108) andperforming communication via the established communication channel. Thecommunication module 190 may include one or more communicationprocessors that are operable independently from the processor 120 (e.g.,the application processor (AP)) and supports a direct (e.g., wired)communication or a wireless communication. According to an embodiment,the communication module 190 may include a wireless communication module192 (e.g., a cellular communication module, a short-range wirelesscommunication module, or a global navigation satellite system (GNSS)communication module) or a wired communication module 194 (e.g., a localarea network (LAN) communication module or a power line communication(PLC) module). A corresponding one of these communication modules maycommunicate with the external electronic device via the first network198 (e.g., a short-range communication network, such as Bluetooth™,wireless-fidelity (Wi-Fi) direct, or infrared data association (IrDA))or the second network 199 (e.g., a long-range communication network,such as a cellular network, the Internet, or a computer network (e.g.,LAN or wide area network (WAN)). These various types of communicationmodules may be implemented as a single component (e.g., a single chip),or may be implemented as multi components (e.g., multi chips) separatefrom each other. The wireless communication module 192 may identify andauthenticate the electronic device 101 in a communication network, suchas the first network 198 or the second network 199, using subscriberinformation (e.g., international mobile subscriber identity (IMSI))stored in the subscriber identification module 196.

The antenna module 197 may transmit or receive a signal or power to orfrom the outside (e.g., the external electronic device) of theelectronic device 101. According to an embodiment, the antenna module197 may include one or more antennas, and, therefrom, at least oneantenna appropriate for a communication scheme used in the communicationnetwork, such as the first network 198 or the second network 199, may beselected, for example, by the communication module 190 (e.g., thewireless communication module 192). The signal or the power may then betransmitted or received between the communication module 190 and theexternal electronic device via the selected at least one antenna.

At least some of the above-described components may be coupled mutuallyand communicate signals (e.g., commands or data) therebetween via aninter-peripheral communication scheme (e.g., a bus, general purposeinput and output (GPIO), serial peripheral interface (SPI), or mobileindustry processor interface (MIPI)).

According to an embodiment, commands or data may be transmitted orreceived between the electronic device 101 and the external electronicdevice 104 via the server 108 coupled with the second network 199. Eachof the electronic devices 102 and 104 may be a device of a same type as,or a different type, from the electronic device 101.

According to an embodiment, all or some of operations to be executed atthe electronic device 101 may be executed at one or more of the externalelectronic devices 102, 104, or 108. For example, if the electronicdevice 101 should perform a function or a service automatically, or inresponse to a request from a user or another device, the electronicdevice 101, instead of, or in addition to, executing the function or theservice, may request the one or more external electronic devices toperform at least part of the function or the service. The one or moreexternal electronic devices receiving the request may perform the atleast part of the function or the service requested, or an additionalfunction or an additional service related to the request, and transferan outcome of the performing to the electronic device 101. Theelectronic device 101 may provide the outcome, with or without furtherprocessing of the outcome, as at least part of a reply to the request.To that end, a cloud computing, distributed computing, or client-servercomputing technology may be used, for example.

The electronic device 101 includes the processor 120, the memory 130,the display 160, the camera 180, and/or the communication circuitry 190.

The camera 180 may include at least one camera module. For example, thecamera 180 may include a plurality of camera modules and may obtain animage using at least one of the plurality of camera modules under thecontrol of the processor 120. The camera 180 may be controlled based ona plurality of parameters associated with photographing. For example,the parameters associated with the photographing may include at leastone of photosensitivity, a lens diameter, an aperture size, a shutterspeed, focus, exposure, hue, a color temperature, or white balance.

The processor 120 may display an image on at least a portion of thedisplay 160. For example, the processor 120 may obtain an image from theelectronic device 104 using the camera 180 or the communicationcircuitry 190.

The electronic device 101 may perform scene recognition. For example,the electronic device 101 may recognize a foreground (e.g., an objectregion) and background of an image, and may perform the scenerecognition based on the recognized foreground and background. Theforeground corresponds to the object region, and the background maycorrespond to an entire region of an image including the object region.Alternatively, the foreground may correspond to the object region, andthe background may correspond to a remaining region of the image exceptfor at least a portion of the object region.

The electronic device 101 may perform the scene recognition based on aspecified condition. For example, the electronic device 101 maydetermine whether to perform scene recognition based on the similaritybetween an image, which is previously obtained, and an image which iscurrently obtained. Alternatively, the electronic device 101 maydetermine whether to perform the scene recognition based on at least oneof a specified time, the state of the camera 180 (e.g., focus andexposure), or the brightness of the obtained image.

The electronic device 101 may adjust an image quality setting based onthe recognized scene. The electronic device 101 may adjust the imagequality setting by adjusting at least one parameter associated with theobtaining an image by the camera 180. For example, the electronic device101 may adjust at least one of photosensitivity, a lens diameter, anaperture size, a shutter speed, focus, exposure, hue, a colortemperature, or white balance of the camera 180.

The electronic device 101 may adjust the image quality setting byadjusting at least one image parameter of the obtained image. Forexample, the electronic device 101 may perform image correction byadjusting the brightness, contrast, gamma, hue, color space, sharpness,blur, or a color temperature of the image.

The electronic device 101 may perform image retrieval based on therecognized scene. For example, the electronic device 101 may retrievethe name of an object when the object of the recognized scenecorresponds to a flower, an animal, a bird, or a tree. As anotherexample, the electronic device 101 may retrieve the name and/or recipeof food when the object of the recognized scene corresponds to the food.When the recognized scene corresponds to a street, the electronic device101 may estimate user context (e.g., path finding) corresponding to therecognized scene, and obtain the information on the current position ofthe electronic device 101 depending on the estimated user context. Asanother example, the electronic device 101 may recognize a textcontained in the image, translate the text, retrieve the text, and/orscan a document.

At least some of the operations of the electronic device 101 describedabove may be performed by an external electronic device 104.

Scene recognition may be performed by the electronic device 104, and theelectronic device 101 may receive the result of the scene recognitionfrom the electronic device 104. The scene recognition may be performedby the external electronic device 104, and the electronic device 101 mayreceive an image quality setting parameter corresponding to the scenerecognition from the external electronic device.

The scene recognition and the image quality setting may be adjusted bythe electronic device 104. For example, the electronic device 101 mayreceive an image, which is corrected based on the adjusted image qualitysetting, from the electronic device 104.

FIG. 2 illustrates a camera module according to an embodiment.

Referring to FIG. 2, the camera module 180 may include a lens assembly210, a flash 220, an image sensor 230, an image stabilizer 240, memory250 (e.g., buffer memory), or an image signal processor 260. The lensassembly 210 may collect light emitted or reflected from an object whoseimage is to be taken. The lens assembly 210 may include one or morelenses. According to an embodiment, the camera module 180 may include aplurality of lens assemblies 210. In such a case, the camera module 180may form, for example, a dual camera, a 360-degree camera, or aspherical camera. Some of the plurality of lens assemblies 210 may havethe same lens attribute (e.g., view angle, focal length, auto-focusing,f number, or optical zoom), or at least one lens assembly may have oneor more lens attributes different from those of another lens assembly.The lens assembly 210 may include, for example, a wide-angle lens or atelephoto lens.

The flash 220 may emit light that is used to reinforce light reflectedfrom an object. According to an embodiment, the flash 220 may includeone or more light emitting diodes (LEDs) (e.g., a red-green-blue (RGB)LED, a white LED, an infrared (IR) LED, or an ultraviolet (UV) LED) or axenon lamp. The image sensor 230 may obtain an image corresponding to anobject by converting light emitted or reflected from the object andtransmitted via the lens assembly 210 into an electrical signal.According to an embodiment, the image sensor 230 may include oneselected from image sensors having different attributes, such as a RGBsensor, a black-and-white (BW) sensor, an IR sensor, or a UV sensor, aplurality of image sensors having the same attribute, or a plurality ofimage sensors having different attributes. Each image sensor included inthe image sensor 230 may be implemented using, for example, a chargedcoupled device (CCD) sensor or a complementary metal oxide semiconductor(CMOS) sensor.

The image stabilizer 240 may move the image sensor 230 or at least onelens included in the lens assembly 210 in a particular direction, orcontrol an operational attribute (e.g., adjust the read-out timing) ofthe image sensor 230 in response to the movement of the camera module180 or the electronic device 101 including the camera module 180. Thisallows compensating for at least part of a negative effect (e.g., imageblurring) by the movement on an image being captured. According to anembodiment, the image stabilizer 240 may sense such a movement by thecamera module 180 or the electronic device 101 using a gyro sensor (notshown) or an acceleration sensor (not shown) disposed inside or outsidethe camera module 180. According to an embodiment, the image stabilizer240 may be implemented, for example, as an optical image stabilizer.

The memory 250 may store, at least temporarily, at least part of animage obtained via the image sensor 230 for a subsequent imageprocessing task. For example, if image capturing is delayed due toshutter lag or multiple images are quickly captured, a raw imageobtained (e.g., a Bayer-patterned image, a high-resolution image) may bestored in the memory 250, and its corresponding copy image (e.g., alow-resolution image) may be previewed via the display device 160.Thereafter, if a specified condition is met (e.g., by a user's input orsystem command), at least part of the raw image stored in the memory 250may be obtained and processed, for example, by the image signalprocessor 260. According to an embodiment, the memory 250 may beconfigured as at least part of the memory 130 or as a separate memorythat is operated independently from the memory 130.

The image signal processor 260 may perform one or more image processingwith respect to an image obtained via the image sensor 230 or an imagestored in the memory 250. The one or more image processing may include,for example, depth map generation, three-dimensional (3D) modeling,panorama generation, feature point extraction,, image synthesizing, orimage compensation (e.g., noise reduction, resolution adjustment,brightness adjustment, blurring, sharpening, or softening). Additionallyor alternatively, the image signal processor 260 may perform control(e.g., exposure time control or read-out timing control) with respect toat least one (e.g., the image sensor 230) of the components included inthe camera module 180. An image processed by the image signal processor260 may be stored back in the memory 250 for further processing, or maybe provided to an external component (e.g., the memory 130, the displaydevice 160, the electronic device 102, the electronic device 104, or theserver 108) outside the camera module 180. According to an embodiment,the image signal processor 260 may be configured as at least part of theprocessor 120, or as a separate processor that is operated independentlyfrom the processor 120. If the image signal processor 260 is configuredas a separate processor from the processor 120, at least one imageprocessed by the image signal processor 260 may be displayed, by theprocessor 120, via the display device 160 as it is or after beingfurther processed.

According to an embodiment, the electronic device 101 may include aplurality of camera modules 180 having different attributes orfunctions. In such a case, at least one of the plurality of cameramodules 180 may form, for example, a wide-angle camera and at leastanother of the plurality of camera modules 180 may form a telephotocamera. Similarly, at least one of the plurality of camera modules 180may form, for example, a front camera and at least another of theplurality of camera modules 180 may form a rear camera.

FIG. 3 illustrates an electronic device, according to an embodiment.

Referring to FIG. 3, the electronic device 101 includes a scenerecognition determination module 301, a scene recognition module 303, ascene determination module 305, a subsidiary scene determination module307, and an image quality processing module 309.

The components of the electronic device 101 in FIG. 3 are provided forillustrative purposes, and are not limited to the components illustratedin FIG. 3. The components of the electronic device 101 illustrated inFIG. 3 may be software modules, e.g., software functions and/or dataproduced as instructions stored in a memory of the electronic device 101that are performed by the processor.

The scene recognition determination module 301 may determine whether toperform scene recognition, based on the obtained image. The scenerecognition determination module 301 may determine whether to performthe scene recognition based on various criteria. The scene recognitiondetermination module 301 may be used to reduce power consumptionresulting from the scene recognition, to maintain the consistency of thescene recognition, and to prevent errors in the scene recognition. Thescene recognition determination module 301 may determine whether toperform the scene recognition, based at least partially on at least oneof a camera state, a timer, image brightness, image similarity, ormotion information of the electronic device 101.

The scene recognition determination module 301 may determine whether toperform the scene recognition, based on the state of the camera. Forexample, the scene recognition determination module 301 may determine toperform the scene recognition, when the camera obtains automaticexposure and automatic focus. The scene recognition determination module301 may perform or determine the scene recognition when the state of theautomatic exposure is converged to a certain exposure level and theautomatic focus is focused.

The scene recognition determination module 301 may determine whether toperform the scene recognition based on the timer. For example, the scenerecognition determination module 301 may determine not to perform thescene recognition, when a specified first time does not elapse from thescene recognition which has been the most lastly performed. In thiscase, the scene recognition determination module 301 may determine toperform the scene recognition, after the specified first time elapsesfrom the scene recognition which has been the most lastly performed.

The scene recognition determination module 301 may determine to performthe scene recognition, when a specified second time elapses from thescene recognition which has been the most lastly performed. In thiscase, when the specified second time has elapsed, the scene recognitiondetermination module 301 may regard the validity period for the previousscene recognition as being expired, and determine to perform new scenerecognition.

The scene recognition determination module 301 may determine whether toperform the scene recognition, based on the motion of the electronicdevice 101. For example, the scene recognition determination module 301may determine not to perform the scene recognition, when the motion ofthe electronic device 101 is greater than or equal to a specific range.The scene recognition determination module 301 may determine to performthe scene recognition, when the motion of the electronic device 101 isless than a specific range. As another example, the scene recognitiondetermination module 301 may determine to perform the scene recognition,when the motion of the electronic device 101 is less than a specificrange for a specific time.

The scene recognition determination module 301 may determine whether toperform the scene recognition, based on the brightness of the image. Forexample, the scene recognition determination module 301 may determinenot to perform scene recognition based on a relevant image, when thebrightness (e.g., the average brightness of the image) of the image isless than a specified value. The scene recognition determination module301 may determine to perform the scene recognition based on a relevantimage, when the brightness (e.g., the average brightness of the image)of the image is greater than or equal to a specified value.

The scene recognition determination module 301 may determine whether toperform the scene recognition, based on the similarity between images.For example, when a scene, which is previously recognized, is presentwithin a specified time, and when the similarity between a previousimage, which is used for recognition of a previous scene, and a presentimage is greater than or equal to a specified value, the scenerecognition determination module 301 may determine not to perform thescene recognition. The scene recognition determination module 301 maydetermine to perform the scene recognition, when the similarity betweenthe previous image and the present image is less than the specifiedvalue.

The conditions for the scene recognition described above may be combinedwith each other. The scene recognition determination module 301 mayidentify the state of the camera 180 when a specified time elapses fromthe last scene recognition. When the state of the camera 180 correspondsto a specified state (e.g., automatic exposure and automatic focusobtained), the scene recognition determination module 301 may obtain theinformation on the motion of the electronic device 101.

When the obtained motion information is less than the specified range,the scene recognition determination module 301 may obtain theinformation on the brightness of the image. When the obtained motioninformation is less than the specified range, the scene recognitiondetermination module 301 may determine whether the state of the camera180 corresponds to a specified state after the specified time elapses.

When the state of the camera 180 corresponds to the specified state, theinformation on the brightness of the image may be obtained. When theobtained information on the brightness of the image is greater than orequal to the specified value, the scene recognition determination module301 may determine the similarity to the previous image. When thesimilarity to the previous image is less than the specified value, thescene recognition determination module 301 may determine to perform thescene recognition.

As another example, when the scene recognition has not been previouslyperformed (e.g., within a specified time), the scene recognitiondetermination module 301 may determine to perform the scene recognition,regardless of the similarity to the previous image.

The scene recognition module 303 includes a whole image analyzer 310 andan object image analyzer 320.

The scene recognition module 303 may analyze an image using the wholeimage analyzer 310 and/or the object image analyzer 320 and may transmitthe analysis result to the scene determination module 305. The scenerecognition module 303 may determine the type (e.g., a scene tag) of ascene, based on the result of the recognition of an object (e.g., ahuman (person), an animal, a flower, or a tree) in the image and thesemantic analysis of the whole image.

The whole image analyzer 310 may determine the type of an image by usinginformation on the whole features of the image. For example, the wholeimage analyzer 310 may obtain the scene type of the image. The wholeimage analyzer 310 may identify a tag (e.g., a semantic tag)corresponding to the whole image. The whole image analyzer 310 maydetermine the scene type of the image using feature information for thebackground region, except for the object region of the image. The scenetype of the image of the whole image analyzer 310 may include at leastone category (e.g., a tag) for describing the scene. The scene type maybe a mountain, a sunrise, a sunset, a landscape, a beach, a sky, snow, anight view, a street, a house, a waterside, a waterfall, a city,greenery, a tree, a flower garden, and/or an invalidity. For example,even if the size of a region of interest is less than a specified size,so the size of the region is not identified as a person, a face, or anobject, the whole image analyzer 310 may detect the type correspondingto the object.

The object image analyzer 320 may identify at least one object from theregion of interest of the image. For example, the object image analyzer320 may identify at least one object from a portion of the image. Theobject image analyzer 320 may identify a tag corresponding to theidentified object.

The object image analyzer 320 includes a face detector 321, a humandetector 323, an object detector 325, and/or an object identifier 327.

The face detector 321 may detect a face from a region of interest. Theface detector 321 may transmit object information (e.g., a tag)corresponding to the face to the scene determination module 305 when aface image satisfying a specified condition is detected from the image.The face detector 321 may determine a face as being detected, when aface in a specified size or more is detected. The face detector 321 maydetermine a face as being detected, when a face image having a specifiedpercentage of the whole image is detected, when the face image (e.g.,the center of the face or the border of the face) is spaced from theboundary of the image by more than a specified distance, or when theface image is positioned at the center of the image. The face detector321 may perform a separate algorithm to distinguish between a face of ahuman and a face of an object (e.g., a mannequin or stone statue).

The human detector 323 may detect a human from a region of interest. Forexample, the human detector 323 may detect a human region correspondingto a human from the region of interest, or may detect a human regionfrom the region of interest and detect the human using a face region inless than a specified size detected by the face detector 321. The humandetector 323 may determine whether the human is detected, based on theoverlap degree of the face region and the human region. The humandetector 323 may transmit object information corresponding to the humanto the scene determination module 305 when the human is detected.

The object detector 325 may identify an object region from the region ofinterest. The object detector 325 may identify the object region byidentifying the region of interest from the image based on informationon the image (e.g., a saliency map and/or an index map). The objectdetector 325 may identify the object region by identifying a boundarycorresponding to the object. The object detector 325 may transmit theinformation on the object region to the object identifier 327 when theidentified object region is greater than or equal to a specified size.The object detector 325 may identify the object region when no face orhuman is detected by the face detector 321 and the human detector 323.When the object region having the specified size or more is notidentified, the object detector 325 may transmit a tag corresponding to“invalid” to the scene determination module 305.

The object identifier 327 may identify an object from the object region.For example, the object identifier 327 may identify an object such as ananimal, a bird, a tree, food, or a flower. The object identifier 327 maytransmit information on the identified object to the scene determinationmodule 305.

The scene determination module 305 may determine a scene (e.g., a tagcorresponding to the scene) based on a scene type (e.g., a tag) from thewhole image analyzer 310 and the object information from the objectimage analyzer 320. When receiving a meaningful face (i.e., a size ofthe face is greater than a predetermined size, a ratio of the face tothe whole image is greater that a predetermined ration, or a reliabilityfor detected face is greater than a predetermined reliability) or humandetection result from the face detector 321 or the human detector 323,the scene determination module 305 may determine the image as a pictureof a human. When the object information is received from the objectidentifier 327, the scene determination module 305 may determine theimage as a picture corresponding to an object, based on the position ofthe object region corresponding to the object and/or the reliability ofthe object. The scene determination module 305 may determine the imageas the picture corresponding to the object, when the position of theobject region is positioned at the center of the image and/or when thereliability for the identified object is greater than or equal to aspecified value. The scene determination module 305 may determine thereliability of the identified image by using a recognition engine (e.g.,a deep network) of the electronic device 101 or an external electronicdevice. The scene determination module 305 may determine the scene ofthe image, based on the scene type of the image analyzed by the wholeimage analyzer 310, when it is determined that the image does notcorrespond to a face, a human, and an object. The scene determinationmodule 305 may determine a scene of an image based on a scene type, whenthe reliability of the scene type by the whole image analyzer 310 ishigher than a specified value.

The scene determination module 305 may analyze the validity of the scenetype analyzed by the whole image analyzer 310. For example, when thescene type corresponding to the night view is detected by the wholeimage analyzer 310, but when the time for obtaining the image is daytimeor in the morning, the scene determination module 305 may ignore thescene type analysis by the whole image analyzer 310.

The scene determination module 305 may determine a scene (e.g., a tagcorresponding to a scene), based on the reliability of the scene type(e.g., a tag) from the whole image analyzer 310 and the reliability ofthe object information (e.g., a tag) from the object image analyzer 320.The reliability of the face or human detected by the object imageanalyzer 320 may be set to be higher than the information on anotherobject. The electronic device 101 may obtain the reliability of thescene type analyzed by the whole image analyzer 310 using a recognitionengine (e.g., a deep network) of the electronic device 101 or theexternal electronic device.

The scene recognition module 303 may use the whole image analyzer 310based on the analysis result of the object image analyzer 320. Forexample, when an effective face region larger than a specified size isdetected by the face detector 321, the scene recognition module 303 maynot use the human detector 323, the object detector 325, the objectidentifier 327, and the whole image analyzer 310. As another example,the scene recognition module 303 may not use the whole image analyzer310 when a face or human is detected by the object image analyzer 320 oran object having more than a specified reliability is detected. Thescene recognition module 303 may determine a scene type of an image byusing the whole image analyzer 310 when the object image analyzer 320fails to detect the face, the human, and the object having specifiedreliability.

The scene recognition module 303 may utilize the whole image analyzer310 independently of the object image analyzer 320. The scenerecognition module 303 may use the object image analyzer 320 and thewhole image analyzer 310 in parallel.

A subsidiary scene determination module 307 may determine a scene of animage, when the scene is not determined by the scene determinationmodule 305. For example, when the scene determination module 305 may notdetermine the scene (e.g., when the face, human, and object are notidentified and the reliability of the scene type is less than thespecified range), the subsidiary scene determination module 307 maydetermine reliability based on the sum of reliabilities for upper-levelcategories of scene types identified by the scene determination module305. A scene type may be identified as a mountain, a waterside, or abeach, based on a picture and the reliability of each of identifiedscene types may be less than a specified value. In this case, thesubsidiary scene determination module 307 may determine a scene as alandscape which is an upper-level category of the mountain, thewaterside, and the beach, when the sum of the reliabilities of themountain, the waterside, and the beach is greater than or equal to aspecified value. The subsidiary scene determination module 307 mayanalyze the validity of the determined scene. For example, thesubsidiary scene determination module 307 may determine the validity ofthe determined scene, based on a time at which the image is obtainedand/or information on a place at which the image is obtained.

The scene determination module 305 may determine the scene using theresult having the highest reliability derived by the whole imageanalyzer 310 and the object image analyzer 320. The subsidiary scenedetermination module 307 may determine the scene using both a result(e.g., a scene type less than specified reliability), which is notconsidered by the scene determination module 305, and the highestreliability result.

Alternatively, the subsidiary scene determination module 307 may beomitted, or the subsidiary scene determination module 307 may beincluded in the scene determination module 305.

The image quality processing module 309 may adjust the image qualitysetting using the scene determined by the scene determination module 305or the subsidiary scene determination module 307 (e.g., a scene tag).The image quality processing module 309 may generate an image subject toan image quality setting, which is adjusted by modifying imageparameters with respect to the obtained image. For example, the imageparameters may include at least one of brightness, contrast, gamma, hue,sharpness, blur, and/or color temperature.

The image quality processing module 309 may adjust the image qualitysetting by adjusting at least one parameter, which is associated withobtaining an image, of the camera 180. The at least one parameter mayinclude at least one of photosensitivity, a lens diameter, an aperturesize, a shutter speed, exposure, focus, a hue, a color temperature,and/or white balance. When no meaningful results are received from thescene determination module 305 and the subsidiary scene determinationmodule 307, the image quality processing module 309 may adjust the imagequality setting using a specified image parameter or the parametersassociated with obtaining the image.

FIG. 4 is a flowchart 400 illustrating a method for determining scenerecognition, according to an embodiment.

For example, the method for determining scene recognition of FIG. 4 maybe an example of determining whether to perform scene recognition by thescene recognition determination module 301 of FIG. 3.

Referring to FIG. 4, in step 405, the electronic device 101 (e.g., theprocessor 120) obtains a first image. For example, the electronic device101 may obtain the first image using a camera. The electronic device 101may adjust the image quality setting for the first image using theobtained first image.

In step 410, the electronic device 101 obtains a second image. Forexample, the electronic device 101 may obtain the second image by usingthe camera 180. The electronic device 101 may obtain the second image ina specified time after the first image is obtained. The electronicdevice 101 may obtain the second image, based on the camera state (e.g.,automatic exposure and automatic focus states) and/or the movementinformation of the electronic device 101.

In step 415, the electronic device 101 determines whether the brightnessof the second image is greater than or equal to a specified range. Forexample, the electronic device 101 may determine the second image as avalid image when the brightness of the second image (e.g., averagebrightness) is in the specified first range. The electronic device 101may determine the second image as an invalid image when the brightnessof the second image is within a second range less than the specifiedfirst range. The electronic device 101 may obtain a new second imagewhen the second image is determined to be the invalid image.

When the brightness of the second image is greater than or equal to thespecified range, the electronic device 101 determines whether thesimilarity between the first image and the second image is greater thanor equal to a specified range in step 420. For example, the electronicdevice 101 may determine the similarity, based on a difference betweenthe first image and the second image, features of the first image andthe second image, and/or the correlation between the first image and thesecond image.

When the similarity between the first image and the second image is in aspecified second range less than the specified first range, theelectronic device 101 determines to perform the scene recognition basedon the second image in step 425.

When the similarity between the first image and the second image is inthe specified first range exceeding the specified second range, theelectronic device 101 determines to not perform the scene recognitionfor the second image. In this case, the electronic device 101 may adjustthe image quality setting for the second image, based on the imagequality setting determined for the first image.

FIG. 5 is a flowchart 500 illustrating a method for determining a typeof a background image, according to an embodiment.

For example, the method for determining the background image type inFIG. 5 may be an example of a method for determining a scene by thewhole image analyzer 310 in FIG. 3.

Referring to FIG. 5, in step 505, the electronic device 101 may identifyat least one tag based on a second image. For example, the electronicdevice 101 may identify at least one tag corresponding to a scene typeof the second image based on the entire image region of the secondimage. The electronic device 101 may determine the at least one tagidentified based on the entire image region and reliabilitycorresponding to the at least one tag. The electronic device 101 maydetermine the reliability of the at least one tag based at least on aratio of an image region associated with the at least one tag.

In step 510, the electronic device 101 transmits the identified tag to ascene determination module 305. The electronic device 101 may transmit,to the scene determination module 305, the information on the at leastone tag and the reliability of the at least one tag.

The electronic device 101 may perform the method for determining thebackground image type of FIG. 5, when the object is not identified basedon the second image.

FIG. 6 is a flowchart 600 illustrating a method for identifying anobject, according to an embodiment.

For example, the method for identifying the object in FIG. 6 may be anexample of a method for identifying an object by the object imageanalyzer 320.

Referring to FIG. 6, in step 605, the electronic device 101 determineswhether a face region is detected from a second image. For example, theelectronic device 101 may determine that the face region is detected,when a valid face region having a predetermined size (e.g., a specifiedratio) is detected from the second image.

In step 607, when a face region is detected in step 605, the electronicdevice 101 transmits a tag corresponding to the detected face region tothe scene determination module 305. For example, the electronic device101 may transmit, to the scene determination module 305, a tagcorresponding to a plurality of face regions for detecting a pluralityof valid face regions having a specified size or more.

When the face region is not detected in step 605, the electronic device101 determines whether a human region is detected in step 610. Forexample, the electronic device 101 may detect the human region byidentifying an object corresponding to the human from the second image.As another example, the electronic device 101 may detect the humanregion based on whether the object corresponding to the human detectedfrom the second image overlaps with the face region (e.g., a face regionhaving less than the specified size). When the human region is detectedin step 610, the electronic device 101 transmits, to the scenedetermination module 305, the tag corresponding to the human region instep 612.

When the human region is not detected in step 610, the electronic device101 determines whether the object is detected from the second image instep 615. For example, when the object region having the specified sizeor more is detected from the second image, the electronic device 101identifies an object corresponding to the object region in step 620. Forexample, the electronic device 101 may identify a boundary of an objectand/or the object based on the feature point of an object region. Instep 625, the electronic device 101 transmits a tag corresponding to theidentified object to the scene determination module 305.

When the object is not detected in step 615 (e.g., when the objectregion having less than the specified size is detected, when thereliability of the detected object region is less than the specifiedreliability, or when the object region is not detected), the electronicdevice 101 may transmit an image type to the scene determination module305 depending on the method for determining the image type, e.g., asdescribed above with reference to FIG. 5.

FIG. 7 is a flowchart 700 illustrating a method for determining a scene,according to an embodiment.

For example, the method for determining the scene in FIG. 7 may be anexample of a method for identifying a scheme by the scene determinationmodule 305 of FIG. 3.

Referring to FIG. 7, the electronic device 101 identifies (ordetermines) at least one scene tag from identified tags in step 705. Forexample, the electronic device 101 may determine the scene tag to a tagcorresponding to the face or the human when the face or the human isidentified. As another example, when an object is identified, theelectronic device 101 may determine the scene tag to a tag correspondingto the identified object based at least on the reliability of theidentified object. As another example, when the tag corresponding to thescene type is identified, the electronic device 101 may determine thetag corresponding to the scene type as a scene tag based on thereliability of the scene type.

In step 710, the electronic device 101 determines whether at least onescene tag is valid. The electronic device 101 may determine the validityof the scene tag based on at least one of time at which the second imageis obtained or a place in which the second image is obtained. Forexample, the electronic device 101 may determine that the scene tag isinvalid when the scene tag does not correspond to the information on thetime at which the second image is obtained. The electronic device 101may determine the scene tag as being invalid when the information on thetime, at which the second image is obtained, indicates lunch, evening,or night even though the scene tag corresponds to morning. As anotherexample, the electronic device 101 may determine the scene tag as beinginvalid when the scene tag does not correspond to the information on theplace in which the second image is obtained. The electronic device 101may determine the scene tag as being invalid when the scene tagcorresponds to sun rising and the place information indicates the Westcoast.

The electronic device 101 may change the scene tag based on the timeinformation and/or the place information when the scene tag isdetermined as being invalid. For example, when the scene tag correspondsto the morning and the information on the time, at which the secondimage is obtained, corresponds to the evening, the electronic device 101may change the scene tag to a tag corresponding to the evening.

In step 715, when the scene tag is determined as being valid, theelectronic device 101 transmits, to the image quality processing module309, the scene tag.

FIG. 8 is a flowchart 800 illustrating a method for subsidiarilydetermining a scene, according to an embodiment.

For example, the method for subsidiarily determining a scene in FIG. 8may be an example of a method for identifying a scene by a subsidiaryscene determining module 307 of FIG. 3.

For example, the electronic device 101 may perform the method forsubsidiarily determining the scene as illustrated in FIG. 8, when thescene tag is invalid or when the identified object or scene type isabsent, when the image type, which is determined through the method fordetermining the image type as illustrated in FIG. 5, is not identifiedor the reliability of the image type is less than a specified value, orwhen the reliability of the identified object is less than the specifiedvalue without identifying the face and human regions through the methodfor identifying the object as illustrated in FIG. 6.

Referring to FIG. 8, the electronic device 101 determines whether thereliabilities of the identified tags are greater than or equal to aspecified range in step 805. For example, the electronic device 101 maydetermine a scene by using only a tag, which has the reliability greaterthan or equal to a specified range, of the identified tags (e.g., a tagcorresponding to a scene type, a tag corresponding to a face region, atag corresponding to a human region, and/or a tag corresponding to theobject region).

In step 810, when the reliabilities of the identified tags are greaterthan or equal to the specified range in step 805, the electronic device101 determines whether at least one identified tag, which has thereliability greater than or equal to the specified range, corresponds tothe information on the time and/or the position at which the image isobtained. For example, the electronic device 101 may determine a sceneusing only tags (e.g., valid tags), which correspond to information onthe time and/or the position, of tags having the reliability greaterthan or equal to the specified range.

In step 815, when at least one identified tag corresponds to theinformation on the time and/or the position at which the image isobtained in step 810, the electronic device 101 transmits, to the imagequality processing module 309 in FIG. 3, at least one scene tagcorresponding to an upper-level category of valid tags, which have aspecified reliability or more, of the identified tags.

The electronic device 101 may set, as a scene tag, a tag correspondingto an upper-level category, which has the highest reliability, of aplurality of upper-level categories, based on the sum of reliabilitiesof the valid tags having the specified reliability or more. For example,the electronic device 101 may identify a mountain (e.g., having 33% inreliability), a flower (e.g., having 34% in reliability), and an animal(e.g., having 20% in reliability) from the second image. For example,the electronic device 101 may set, as a scene tag of the second image,woods which are a common upper-level category of the mountain and theflower, based on the sum of the reliabilities of the mountain and theflower.

FIG. 9 is a flowchart 900 illustrating a method for determining an imagequality setting, according to an embodiment.

For example, the method for adjusting the image quality setting of FIG.9 may be an example of a method for adjusting an image quality settingby the image quality processing module 309 of FIG. 3.

Referring to FIG. 9, in step 905, the electronic device 101 obtains atleast one scene tag. For example, the electronic device 101 may obtain ascene tag set through the method for determining a scene in FIG. 7 orthe method for subsidiary determining the scene in FIG. 8

In step 910, the electronic device 101 obtains at least one imagequality setting parameter corresponding to information on the scene tag.For example, the image quality setting parameter may include at leastone of brightness, contrast, gamma, hue, sharpness, blur, or a colortemperature. As another example, the image quality setting parameter mayinclude at least one of photosensitivity, a lens diameter, an aperturesize, a shutter speed, exposure, focus, hue, color temperature, or whitebalance. The electronic device 101 may transmit the information on thescene tag to the electronic device 104 of FIG. 1, and may obtain theimage quality setting parameter from the external electronic device.

In step 915, the electronic device 101 sets the image quality of thesecond image based on the obtained image quality setting parameters. Forexample, the electronic device 101 may adjust the image quality settingparameters of the second image obtained using the obtained image qualitysetting parameters. As another example, the electronic device 101 maychange the photographing parameters of the camera by using the obtainedimage quality setting parameters, thereby setting the image quality ofthe second image obtained by the camera 180.

In step 920, the electronic device 101 provides a second image to adisplay. For example, the second image displayed on the display may bean image obtained depending on the image quality setting adjusted basedon the image quality setting parameters.

FIG. 10 illustrates a flow chart 1000 of a method for recognizing ascene, according to an embodiment.

Referring to FIG. 10, the electronic device 101 may perform scenerecognition for an obtained image 1010.

The electronic device 101 may extract an object region image 1020 fromthe obtained image 1010 and may identify the object from the objectregion image 1020. For example, the electronic device 101 may identifythe object region as a flower having 90% in reliability, a tree having30% in reliability, or foods having 4% in reliability.

The electronic device 101 may identify a scene type based on an entireimage 1030 of the obtained image 1010. For example, the electronicdevice 101 may identify the entire image 1030 as a sky having 33% inreliability.

The electronic device 101 may determine a scene corresponding to theobtained image 1010 based on the reliabilities of the identified objectand the identified scene type. For example, when an object is present,which has specified reliability or more, among the object regions, thescene may be determined depending on the object having the highestreliability. For example, the electronic device 101 may determine thescene of the image 1010, which is obtained, as “flower” having thehighest reliability.

FIG. 11 illustrates an image 1100 for determining a reliability,according to an embodiment.

Referring to FIG. 11, the electronic device 101 may obtain the image1100 corresponding to a sunrise by using the camera 180 of theelectronic device 101. For example, the electronic device 101 maydetermine the image 1100 as a scene corresponding to a sunset. Theelectronic device 101 may identify the determined scene as being invalidwhen the information on time, at which the image is obtained, does notindicate the time corresponding to the sunset. When the information on aplace at which the image is obtained corresponds to the East coast, theelectronic device 101 may identify the determined scene, whichcorresponds to the sunset, as being invalid.

FIG. 12 illustrates an image for recognizing an object, according anvarious embodiment.

Referring to FIG. 12, the electronic device 101 may detect face regionsfrom a first image 1200 and a second image 1210. The electronic device101 may determine whether a valid face region has been detected, basedat least on the size and the position of the detected face region. Forexample, the electronic device 101 may determine whether a face regionis detected, based at least on the size and the position of the detectedface region, even if the face region is detected.

The electronic device 101 may recognize the face region from the firstimage 1200. When the position of the face region is at the boundary ofthe first image 1200, the electronic device 101 may not correctly detectthe face region. For example, when the position of the face region ispositioned at the boundary of the first image 1200, a user may haveobtained the first image 1200 without intentionally capturing an imageof a human corresponding to the face region. For example, the humancontained in the first image 1200 may be a human (e.g., a passerby) thatdoes not meet the intent of the user. In this case, the electronicdevice 101 may determine a scene based on a scene type (e.g., a sunrise)rather than the face region.

The electronic device 101 may recognize the face region from the secondimage 1210. For example, the human of the second image 1210 may bepositioned at the center of the second image 1210 or at more than aspecified distance from the boundary of the second image 1210. In thiscase, the electronic device 101 may determine the scene based on therecognized face region.

FIG. 13 illustrates an image for determining a scene, according to anembodiment.

Referring to 13, the electronic device 101 may not detect a face regionand a human region from a first image 1300 and a second image 1310.

The electronic device 101 may identify an object “flower” from the firstimage 1300. The electronic device 101 may determine the identified“flower” as the scene of the first image 1300. For example, theelectronic device 101 may determine the flower corresponding to theidentified object as the scene of the first image 1300, when the size(e.g., a ratio based on the entire image) of the identified object has aspecified size or more.

The electronic device 101 may identify “flower”, which is a scene type,from the second image 1310. In the second image 1310, flowers may havean object region in size less than a specified size. Accordingly, theelectronic device 101 may not identify the “flower” based on objectrecognition from the object region. In this case, the electronic device101 may identify “flower”, which is the scene type, based on the entireregion of the second image 1310. The electronic device 101 may identifythe scene of the second image 1310 as “flower” based on the reliabilityof the identified scene type. For example, the electronic device 101 maydetermine the reliability of the scene type identified based on the sizeof the region corresponding to “flower” of the second image 1310.

FIG. 14 illustrates an image 1400 for determining a scene, according toan embodiment.

Referring to FIG. 14, it is assumed that the electronic device 101 failsto recognize an object (e.g., a face, a human, and/or an object) fromthe image 1400. For example, the electronic device 101 may determine ascene based on the scene type identified from the image 1400.

The electronic device 101 may identify a plurality of scene types fromthe image 1400. For example, the electronic device 101 may identify thesky from a first region 1410, a city from a second region 1420, awaterside from a third region 1430, and a mountain from a fourth region1440. The first region 1410, the second region 1420, the third region1430, and the fourth region 1440 may be similar to each other in size,within the image 1400. In this case, the reliabilities for the firstregion 1410, the second region 1420, the third region 1430, and thefourth region 1440 may be similar to each other. For example, becausethere is no dominant region in the image 1400 of the first region 1410,the second region 1420, the third region 1430, and the fourth region1440, the scene determination module 305 of FIG. 3 may determine thatthe reliabilities of all scene types are less than or equal to aspecified reliability. Thus, for example, the scene determination module305 of the electronic device 101 may not determine the image 1400 as onespecific scene.

The electronic device 101 may determine an upper-level category of theidentified scene type as the scene of the image 1400. For example, theelectronic device 101 may determine, as the scene of the image 1400, oneof upper-level categories of the identified scene type. For example, theelectronic device 101 may determine the upper-level category as thescene of the image 1400, based at least on the sum of reliabilities ofthe scene types belonging to the upper-level category. When the sum ofthe reliabilities of the scene types belonging to the upper-levelcategory is greater than or equal to the specified value, the electronicdevice 101 may determine the upper-level category as the scene of theimage 1400.

The electronic device 101 may determine the landscape, which belongs tothe upper-level category of the sky, a mountain, a waterside, and acity, as a scene of the image 1400. For example, the electronic device101 may determine the scene of the image 1400 as a landscape by usingthe subsidiary scene determination module 307 of FIG. 3.

FIG. 15 is a flowchart 1500 illustrating a method for adjusting an imagequality setting, according to an embodiment.

Referring to FIG. 15, the electronic device 101 includes the camera 180,the memory 130, and the processor 120, which is operatively coupled tothe camera 180 and the memory 130. For example, the processor 120 mayperform the operations of the electronic device 101 described below.

In step 1505, the electronic device 101 (e.g., the processor 120)obtains a plurality of images. For example, the electronic device 101may obtain a plurality of images for one or more external objects byusing the camera 180.

In step 1510, the electronic device 101 identifies a region of interestand a background region by using one or more images of the plurality ofimages when the similarity between a previous image, which is used todetermine the scene before a plurality of images are obtained, and oneor more images of the plurality of images is less than a specifiedrange. As another example, the electronic device 101 may generate one ormore corrected images, which are obtained by correcting at least some ofa plurality of images through a correction scheme set for a previousimage, when the similarity between the previous image, which is used todetermine the scene before a plurality of images are obtained, and oneor more images of the plurality of images is in the specified range.

In step 1510, the electronic device 101 identifies a region of interestand a background region from one or more images while a plurality ofimages are obtained. For example, the electronic device 101 may identifythe region of interest and the background region when scene recognitionis determined by the scene recognition determination module 301. Forexample, the electronic device 101 may identify the background region byusing the whole image analyzer 310 and may identify the region ofinterest by using the object detector 325 of the object image analyzer320.

In step 1515, the electronic device 101 identifies at least one firstobject from the region of interest. For example, the electronic device101 may identify the first object corresponding to the region ofinterest by using the object identifier 327.

In step 1520, the electronic device 101 identifies the type of thebackground region based on at least one second object included in thebackground region. For example, the electronic device 101 may identifythe type of the background region by using the whole image analyzer 310.

In step 1525, the electronic device 101 determines a scene correspondingto the one or more images based at least on the first object and thetype of the background region. The electronic device 101 may determine ascene based on the reliability of the first object and the type. Forexample, the electronic device 101 may determine a scene correspondingto one or more images based on the reliability of the type of thebackground region when the reliability of the first object is less thana specified first value. The electronic device 101 may determine, as ascene corresponding to one or more images, a scene in the upper-levelcategory including at least one type, based on the sum of thereliability of the at least one type corresponding to at least onesecond object included in the background region, when the reliability ofthe type is less than the second value.

The electronic device 101 may determine the type of the scene, which isdetermined based on the reliability of the type, based on at least oneof time information or the position information of the electronic device101. For example, the electronic device 101 may determine the sceneusing the scene determination module 305 and/or the subsidiary scenedetermination module 307.

In step 1530, the electronic device 101 adjusts an image quality settingof one or more images of the plurality of images by using the imagequality setting corresponding to the scene. For example, the electronicdevice 101 may adjust the image quality setting corresponding to thescene by using the image quality processing module 309. The electronicdevice 101 may generate images corresponding to image quality settingsadjusted by modifying at least one image quality setting parameter forthe plurality of images. The at least one image quality settingparameter may include at least one of brightness, contrast, gamma, hue,sharpness, blur, or a color temperature.

According to an embodiment, the electronic device 101 (e.g., theprocessor 120 of FIG. 1) may be configured to generate an imagecorresponding to image quality setting adjusted by adjusting at leastone parameter associated with the obtaining of an image by the camera180. The at least one parameter associated with the obtaining of animage by the camera 180 may include at least one of photosensitivity, alens diameter, an aperture size, a shutter speed, exposure, focus, hue,a color temperature, or white balance.

The electronic device 101 (e.g., the processor 120 of FIG. 1) may beconfigured to display, in the form of a preview, an image, which isobtained by using the camera 180 adjusted by using the at least oneimage quality setting parameter, on the display 160.

According to an embodiment, an electronic device (e.g., the electronicdevice 101 of FIG. 1) may include a camera (e.g., the camera 180 of FIG.1), a memory (e.g., the memory 130 of FIG. 1), and a processor (e.g.,the processor 120 of FIG. 1)that is operatively connected with thecamera and the memory. The processor may be configured to obtain aplurality of images for one or more external objects by using thecamera, to identify at least one region of interest and at least onebackground region by using one or more images of the plurality ofimages, while at least obtaining the plurality of images, to recognizeat least one first object, which is included in the region of theinterest, of the one or more external objects, to identify a type of thebackground region by recognizing at least one second object, which isincluded in the background region, of the one or more external objects,to determine a scene corresponding to the one or more images, based atleast on the at least one first object, which is recognized, and thetype, and to adjust at least one of an image quality setting associatedwith the camera or image quality setting associated with the one or moreimages by using specified image quality setting corresponding to thescene.

The processor may be configured to identify the at least one region ofinterest and the at least one background region by using the one or moreimages of the plurality of images, based on that similarity between aprevious image, which is used to determine the scene before obtainingthe plurality of images, and the one or more images satisfies aspecified first range.

The processor may be configured to adjust the at least one of the imagequality setting associated with the camera or the image quality settingassociated with the one or more images, by using an image qualitysetting set for the previous image, based on that the similarity betweenthe previous image and the one or more images satisfies a specifiedsecond range different from the first range.

The processor may be configured to determine the scene corresponding tothe one or more images, based on reliability of the at least one firstobject which is recognized.

The processor may be configured to determine the scene corresponding tothe one or more images, based on reliability of the type, when thereliability of the at least one first object which is recognized is lessthan a first value.

The processor may be configured to determine, when the reliability ofthe type is less than a specified second value, an upper-level categoryincluding at least one type corresponding to the at least one secondobject included in the background region, based on a sum of reliabilityof the at least one type, and determine the determined upper-levelcategory as the scene corresponding to the one or more images.

The processor may be configured to determine validity of the scene,which is determined based on the reliability of the type, based on atleast one of time information or position information of the electronicdevice.

The image quality setting associated with the one or more images mayinclude at least one of brightness, contrast, gamma, hue, sharpness,blur, or a color temperature.

The image quality setting associated with the camera may include atleast one of photosensitivity, a lens diameter, an aperture size, ashutter speed, exposure, focus, hue, a color temperature, or whitebalance.

The electronic device may further include a display. For example, theprocessor may be configured to display, in a preview form, an image,which is obtained by using the camera adjusted by using the imagequality setting, on the display.

According to an embodiment, an electronic device (e.g., the electronicdevice 101 of FIG. 1) may include a display (e.g., the display device160 of FIG. 1), a camera (e.g., the camera 180 of FIG. 1), a processor(e.g., the processor 120 of FIG. 1) that is operatively connected withthe camera and the display, and a memory (e.g., the memory 130 ofFIG. 1) that is operatively connected with the processor. The memory maystore instructions that when executed, cause the processor to obtain animage, to identify at least one object from a partial region of theobtained image, to identify at least one first tag corresponding to theidentified at least one object, to identify at least one second tagcorresponding to an entire region of the obtained image, to identify ascene tag corresponding to the obtained image, based on reliabilities ofthe at least one first tag and the at least one second tag, and toadjust at least one image quality setting parameter, which is associatedwith obtaining an image, of the camera by using at least one imagequality setting parameter corresponding to the scene tag.

The instructions may cause, when executed, the processor to identify theat least one object, the at least one first tag, and the at least onesecond tag, when similarity between a previous image, which is obtainedbefore the obtained image, and the obtained image is less than aspecified range.

The instructions may cause, when executed, the processor to adjust theat least one of an image quality setting parameter, which is associatedwith obtaining an image, of the camera by using an image quality settingparameter set for the previous image, when the similarity between theprevious image and the obtained image is within the specified range.

The instructions may cause, when executed, the processor to identify, asthe scene tag, a tag corresponding to one upper-level category in atleast one upper-level category, based on a sum of reliabilities ofsecond tags, which belong to the same upper-level category of the atleast one upper-level category of the at least one second tag, whenreliability of the at least one first tag and reliability of the atleast one second tag are less than specified reliability.

The instructions may cause, when executed, the processor to determinevalidity of the scene tag, based on at least one of time information orposition information of the electronic device, and to adjust the atleast one image quality setting parameter, which is associated withobtaining the image, of the camera by using the at least one imagequality setting parameter corresponding to the scene tag, when the scenetag is determined as being valid.

The instructions may cause, when executed, the processor to identify,when a specified tag of the at least one first tag is present, the scenetag based on the specified tag without identifying the at least onesecond tag.

The at least one image quality setting parameter, which is associatedwith obtaining the image, of the camera includes at least one ofphotosensitivity, a lens diameter, an aperture size, a shutter speed,exposure, focus, hue, a color temperature, or white balance.

The instructions may cause, when executed, the processor to display, ina preview form, an image, which is obtained by using the camera adjustedby using the at least one image quality setting parameter, on thedisplay.

According to an embodiment, a method for setting image quality in anelectronic device may include obtaining an image, identifying at leastone object from a partial region of the obtained image, identifying atleast one first tag corresponding to the identified at least one object,identifying at least one second tag corresponding to an entire region ofthe obtained image, identifying a scene tag corresponding to theobtained image, based on reliabilities of the at least one first tag andthe at least one second tag, and adjusting at least one image qualitysetting parameter, which is associated with obtaining an image, by usingat least one image quality setting parameter corresponding to the scenetag.

The at least one image quality setting parameter, which is associatedwith obtaining the image, may include at least one of photosensitivity,a lens diameter, an aperture size, a shutter speed, exposure, focus,hue, a color temperature, or white balance.

The electronic device according to various embodiments may be one ofvarious types of electronic devices. The electronic devices may include,for example, a portable communication device (e.g., a smart phone), acomputer device, a portable multimedia device, a portable medicaldevice, a camera, a wearable device, or a home appliance. According toan embodiment of the disclosure, the electronic devices are not limitedto those described above.

It should be appreciated that various embodiments of the presentdisclosure and the terms used therein are not intended to limit thetechnological features set forth herein to particular embodiments andinclude various changes, equivalents, or replacements for acorresponding embodiment. With regard to the description of thedrawings, similar reference numerals may be used to refer to similar orrelated elements. It is to be understood that a singular form of a nouncorresponding to an item may include one or more of the things, unlessthe relevant context clearly indicates otherwise. As used herein, m eachof such phrases as “A or B,” “at least one of A and B,” “at least one ofA or B,” “A, B, or C”, “at least one of A, B, and C,” and “at least oneof A, B, or C,” may include all possible combinations of the itemsenumerated together in a corresponding one of the phrases. As usedherein, such terms as “1^(st)” and “2^(nd),” or “first” and “second” maybe used to simply distinguish a corresponding component from another,and does not limit the components in other aspect (e.g., importance ororder). It is to be understood that if an element (e.g., a firstelement) is referred to, with or without the term “operatively” or“communicatively”, as “coupled with,” “coupled to,” “connected with,” or“connected to” another element (e.g., a second element), it means thatthe element may be coupled with the other element directly (e.g.,wiredly), wirelessly, or via a third element.

As used herein, the term “module” may include a unit implemented inhardware, software, or firmware, and may interchangeably be used withother terms, for example, “logic,” “logic block,” “part,” or“circuitry”. A module may be a single integral component, or a minimumunit or part thereof, adapted to perform one or more functions. Forexample, according to an embodiment, the module may be implemented in aform of an application-specific integrated circuit (ASIC).

Various embodiments as set forth herein may be implemented as software(e.g., the program 140) including one or more instructions that arestored in a storage medium (e.g., internal memory 136 or external memory138) that is readable by a machine (e.g., the electronic device 101).For example, a processor (e.g., the processor 120) of the machine (e.g.,the electronic device 101) may invoke at least one of the one or moreinstructions stored in the storage medium, and execute it, with orwithout using one or more other components under the control of theprocessor. This allows the machine to be operated to perform at leastone function according to the at least one instruction invoked. The oneor more instructions may include a code generated by a complier or acode executable by an interpreter. The machine-readable storage mediummay be provided in the form of a non-transitory storage medium. Wherein,the term “non-transitory” simply means that the storage medium is atangible device, and does not include a signal (e.g., an electromagneticwave), but this term does not differentiate between where data issemi-permanently stored in the storage medium and where the data istemporarily stored in the storage medium.

According to an embodiment, a method according to various embodiments ofthe disclosure may be included and provided in a computer programproduct. The computer program product may be traded as a product betweena seller and a buyer. The computer program product may be distributed inthe form of a machine-readable storage medium (e.g., compact disc readonly memory (CD-ROM)), or be distributed (e.g., downloaded or uploaded)online via an application store (e.g., Play Store™), or between two userdevices (e.g., smart phones) directly. If distributed online, at leastpart of the computer program product may be temporarily generated or atleast temporarily stored in the machine-readable storage medium, such asmemory of the manufacturer's server, a server of the application store,or a relay server.

According to various embodiments, each component (e.g., a module or aprogram) of the above-described components may include a single entityor multiple entities. According to various embodiments, one or more ofthe above-described components may be omitted, or one or more othercomponents may be added. Alternatively or additionally, a plurality ofcomponents (e.g., modules or programs) may be integrated into a singlecomponent. In such a case, according to various embodiments, theintegrated component may still perform one or more functions of each ofthe plurality of components in the same or similar manner as they areperformed by a corresponding one of the plurality of components beforethe integration. According to various embodiments, operations performedby the module, the program, or another component may be carried outsequentially, in parallel, repeatedly, or heuristically, or one or moreof the operations may be executed in a different order or omitted, orone or more other operations may be added.

As described above, according to various embodiments disclosed in thedisclosure, a method for adjusting an image quality settingcorresponding to the context of an obtained image may be provided.

According to various embodiments disclosed in the disclosure, the powerconsumption may be reduced by obtaining image setting parameters basedon the similarity of images.

Besides, a variety of effects directly or indirectly understood throughthe disclosure may be provided.

While the disclosure has been shown and described with reference tovarious embodiments thereof, it will be understood by those skilled inthe art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. An electronic device comprising: a camera; amemory; and a processor operatively connected with the camera and thememory, wherein the processor is configured to: obtain, using thecamera, a plurality of images for one or more external objects, identifya region of interest and a background region by using one or more imagesof the plurality of images, while obtaining the plurality of images,recognize a first object included in the region of the interest,identify a type of the background region by recognizing a second objectincluded in the background region, determine a scene corresponding tothe one or more images, based on the recognized first object and theidentified type of the background region, and adjust at least one of animage quality setting associated with the camera or an image qualitysetting associated with the one or more images by using a specifiedimage quality setting corresponding to the determined scene.
 2. Theelectronic device of claim 1, wherein the processor is furtherconfigured to: identify the region of interest and the background regionby using the one or more images, when a similarity between a previousimage used to determine the scene before obtaining the plurality ofimages and the one or more images falls within a specified first range.3. The electronic device of claim 2, wherein the processor is furtherconfigured to: adjust the at least one of the image quality settingassociated with the camera or the image quality setting associated withthe one or more images, by using an image quality setting set for theprevious image, when the similarity between the previous image and theone or more images falls within a specified second range, which isdifferent from the first range.
 4. The electronic device of claim 1,wherein the processor is further configured to determine the scenecorresponding to the one or more images based on a reliability of therecognized first object.
 5. The electronic device of claim 4, whereinthe processor is further configured to determine the scene correspondingto the one or more images based on a reliability of the type, when thereliability of the recognized first object is less than a first value.6. The electronic device of claim 5, wherein the processor is furtherconfigured to: determine, when the reliability of the type is less thana specified second value, an upper-level category including at least onetype corresponding to the second object included in the backgroundregion, based on a sum of reliability of the at least one type, anddetermine the determined upper-level category as the scene correspondingto the one or more images.
 7. The electronic device of claim 5, whereinthe processor is further configured to determine a validity of thescene, which is determined based on the reliability of the type, basedon at least one of time information or position information of theelectronic device.
 8. The electronic device of claim 1, wherein theimage quality setting associated with the one or more images comprisesat least one of brightness, contrast, gamma, hue, sharpness, blur, orcolor temperature.
 9. The electronic device of claim 1, wherein theimage quality setting associated with the camera comprises at least oneof photosensitivity, a lens diameter, an aperture size, a shutter speed,exposure, focus, hue, a color temperature, or white balance.
 10. Theelectronic device of claim 9, further comprising a display, wherein theprocessor is further configured to display an image obtained using thecamera adjusted with the image quality setting as a preview on thedisplay.
 11. An electronic device, comprising: a camera; a display; aprocessor operatively connected with the camera and the display; and amemory operatively connected with the processor, wherein the memorycomprises instructions, which when executed, cause the processor to:obtain an image, identify an object from a partial region of theobtained image, identify a first tag corresponding to the identifiedobject, identify a second tag corresponding to an entire region of theobtained image, identify a scene tag corresponding to the obtained imagebased on reliabilities of the first tag and the second tag, and adjustan image quality setting parameter of the camera using an image qualitysetting parameter corresponding to the identified scene tag.
 12. Theelectronic device of claim 11, wherein the instructions, when executed,further cause the processor to identify the object, the first tag, andthe second tag, when a similarity between a previous image obtainedbefore the obtained image and the obtained image falls within aspecified range.
 13. The electronic device of claim 12, wherein theinstructions, when executed, further cause the processor to adjust theimage quality setting parameter of the camera by using an image qualitysetting parameter set for the previous image, when the similaritybetween the previous image and the obtained image falls within thespecified range.
 14. The electronic device of claim 11, wherein theinstructions, when executed, further cause the processor to identify, asthe scene tag, a tag corresponding to one upper-level category in anupper-level category, based on a sum of reliabilities of second tags,which belong to a same upper-level category in the upper-level categoryof the second tag, when a reliability of the first tag and a reliabilityof the second tag are less than a specified reliability.
 15. Theelectronic device of claim 11, wherein the instructions, when executed,further cause the processor to: determine a validity of the scene tag,based on time information or position information of the electronicdevice, and adjust the image quality setting parameter of the camera byusing the image quality setting parameter corresponding to the scenetag, when the scene tag is determined as being valid.
 16. The electronicdevice of claim 11, wherein the instructions, when executed, furthercause the processor to identify, when a specified tag is included in thefirst tag, the scene tag based on the specified tag, without identifyingthe second tag.
 17. The electronic device of claim 11, wherein the imagequality setting parameter of the camera comprises at least one ofphotosensitivity, a lens diameter, an aperture size, a shutter speed,exposure, focus, hue, a color temperature, or white balance.
 18. Theelectronic device of claim 11, wherein the instructions, when executed,further cause the processor to display, on the display, in a previewform, an image obtained using the camera adjusted with the image qualitysetting parameter.
 19. A method for setting image quality in anelectronic device, the method comprising: obtaining an image;identifying an object from a partial region of the obtained image;identifying a first tag corresponding to the identified object;identifying a second tag corresponding to an entire region of theobtained image; identifying a scene tag corresponding to the obtainedimage, based on reliabilities of the first tag and the second tag; andadjusting an image quality setting parameter of a camera of theelectronic device by using an image quality setting parametercorresponding to the scene tag.
 20. The method of claim 19, wherein theimage quality setting parameter of the camera of the electronic devicecomprises at least one of photosensitivity, a lens diameter, an aperturesize, a shutter speed, exposure, focus, hue, a color temperature, orwhite balance.